Papers with scoring functions
Reliable Evaluation Protocol for Low-Precision Retrieval (2026.acl-short)
Copied to clipboard
| Challenge: | Recent studies have shown that low-precision methods can improve performance, but they introduce high variability in the results based on tie resolution. |
| Approach: | They propose a retrieval evaluation protocol designed to reduce tie variation . high-precision scoring and tie-aware retrieval metrics are proposed to reduce this variability . |
| Outcome: | The proposed retrieval evaluation protocol reduces tie-induced instability and recovers expected scores and ranges on 12 retrieval datasets. |
Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs (2025.findings-naacl)
Copied to clipboard
Duygu Nur Yaldiz, Yavuz Faruk Bakman, Baturalp Buyukates, Chenyang Tao, Anil Ramakrishna, Dimitrios Dimitriadis, Jieyu Zhao, Salman Avestimehr
| Challenge: | Existing methods for probability-based UE are limited by their inability to handle biased probabilities and complex semantic dependencies between tokens. |
| Approach: | They propose a learning-based scoring function that captures complex dependencies between tokens and probabilities and produces more reliable responses. |
| Outcome: | The proposed function outperforms existing scoring functions in question-answering and arithmetical reasoning tasks with different datasets. |
ConRPG: Paraphrase Generation using Contexts as Regularizer (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for paraphrase generation lack reliable supervision signals. |
| Approach: | They propose an unsupervised paradigm for paraphrase generation based on contextual language models, candidate filtering and paraphrase model training based upon the selected candidates. |
| Outcome: | The proposed paradigm outperforms existing paraphrase generation methods in supervised and unsupervised setups. |
Ferret: Faster and Effective Automated Red Teaming with Reward-Based Scoring Technique (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing red-teaming methods generate adversarial attacks to identify vulnerabilities, but they face slow performance, limited categorical diversity, and high resource demands. |
| Approach: | They propose a method that generates multiple adversarial prompt mutations per iteration and ranks them using scoring functions. |
| Outcome: | The proposed method achieves a 95% attack success rate and reduces time to a 90% ASR by 15.2%. |
Believing without Seeing: Quality Scores for Contextualizing Vision-Language Model Explanations (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work has found that explanations can easily convince users that inaccurate VLM predictions are correct. |
| Approach: | They propose to evaluate two complementary qualities of VLM-generated explanations via two quality scoring functions to improve their accuracy. |
| Outcome: | The proposed explanations improve accuracy on the A-OKVQA, VizWiz, and MMMU-Pro tasks by 11.1%, including a 15.4% reduction in falsely believing incorrect predictions. |
Zero-Shot Multimodal Retrieval with Multi-Scale Contextual Representations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to multimodal information retrieval (MMIR) lack generalization across different modalities and require annotated training data. |
| Approach: | They propose a fine-tuning-free, two-stage MMIR approach that couples efficient candidate filtering with fine-grained multimodal re-ranking. |
| Outcome: | The proposed approach outperforms supervised methods on 23 datasets. |